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Abstract 

Graphical sequencers have limits in their use as live 
performance tools. It is hypothesized that those 
limits can be ovecome through live coding or text- 
based interfaces. Using a general purpose program¬ 
ming language has advantages over that of a domain- 
specific language. However, a barrier for a musician 
wanting to use a general purpose language for com¬ 
puter music has been the lack of high-level music- 
specific abstractions designed for realtime manipu¬ 
lation, such as those for time. A library for Haskell 
was developed to give computer musicians a high- 
level interface for a heterogenous output enviroment. 
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1 Introduction 

In this paper, a usability problem of live com¬ 
puter music will be briefly examined, and the 
solution of using a general purpose program¬ 
ming language as a shell for music will be pre¬ 
sented. The necessary components created by 
other developers which were used will be intro¬ 
duced. A library called Conductive 1 , developed 
to make a Haskell interpreter into such a shell, 
will then be described in some detail. A con¬ 
ceptual example and some actual code examples 
will be presented. Finally, conclusions reached 
through the development of this library will be 
presented. 

2 The Problem 

Graphical sequencers are poor tools for live 
musical performance in the judgement of this 
author. Users interact with them primarily 
through a mouse or a limited number of key¬ 
board shortcuts and allow limited customiza- 
tions to the manner in which they are con¬ 
trolled. Previous experiences with GUI tools in 
performance showed them to be inflexible and 

1 http://www.renickbell.net/conductive/ 


awkward when trying to execute complex sets 
of parameter changes simultaneously. 

This problem is exacerbated when consider¬ 
ing the wide variety of synths which exist. Mu¬ 
sicians would like to use them together freely, 
but coordinating them is difficult. For use with 
graphical sequencers, synths employ separate 
GUIs which are almost always point-and-click 
and thus cannot be easily manipulated simulta¬ 
neously with other parameters. 

One possible solution to this problem may be 
the use of a programming language as a tool 
for live coding of music or as a text-based in¬ 
terface [Collins et al., 2003]. A general purpose 
programming language which has abstractions 
for music can serve as a control center or shell 
for a heterogeneous set of outputs. In text, the 
user can write out complex parameter changes 
and then execute them simultaneously. A wide 
variety of existing programming tools and text- 
manipulation utilities can be used to make this 
process more efficient. 

Computer music languages, such as Super- 
Collider [McCartney, 2010] and Chuck [Wang, 
2004] exist. McCartney, the creator of Super- 
Collider, says a specialized language for com¬ 
puter music isn’t necessary, but general pur¬ 
pose programming languages aren’t ready yet 
practically [Mccartney, 2002], Musicians man¬ 
age to make music with domain-specific tools, 
but those are unsatisfactory in many ways, such 
as lack of libraries and development tools and 
slow performance. 

General purpose programming languages 
have libraries for dealing with music, but they 
are particularly limited in number regarding 
real-time music systems. Some which al¬ 
ready have such capabilities through libraries 
include Scheme through Impromptu [Sorensen 
and Brown, 2007] or Lua through LuaAV 
[Wakefield and Smith, 2007]. 

The Haskell programming language was seen 
as a good candidate because of several factors: 



3 test-101228.hs 

4 created: Tue Dec 28 22:25:11 JST 2010 

5 

6 ---- 

7 

8 :set -fobject-code -fforce-recomp 

9 :load Conductive.hs 

10 defaultSCGroup 

11 e <- defaultMusicalEnvironment 

12 s <- initializeSampler "/home/renick/audio/ragalike/drum-tests-101224/*" e 

13 deletePlayer e "default" 

14 patterns <- patternslnit e "test-" [10..40] 4 [0.25 0.5..1] [2,3,3,3,4] [0] 

15 newPatternsFromList e patterns [10..40] [2,3,3,3,4] [("more-dense-",4,[0.25,0.5..1.5],[0]),("roff-sparse",8,[1.5,1.75..5],[0,0.25.. 
1-5]),("roff-dense",16,([0.125,0.25..0.75]++[0.25,0.5..2.75]), [0 0.25..1.5])] 

_ 15,1 6% 

[0] |:vim:test-101228.hs* 1:bash- "snare" 16:50 29-Dec-10 

Loading package hosc-0.8 ... linking ... done. 

Loading package split-0.1.2.3 ... linking ... done. 

Loading package hsc3-0.8 ... linking ... done. 

Loading package conductive-hsc3-0.1.1 ... linking ... done. 

*Conductive> e <- defaultMusicalEnvironment 

*Conductive> s <- initializeSampler "/home/renick/audio/ragalike/drum-tests-101224/*" e 
deletePlayer e "default" 

Datterns <- patternslnit e "test-" [10..40] 4 [0.25,0.5. .1] [2,3,3,3,4] [0] 

‘’Conductive:* deletePlayer e "default" 
defaultEnvironment 

*Conductive> patterns <- patternslnit e "test-" [10..40] 4 [0.25,0.5..1] [2,3,3,3,4] [0] 

Loading package MutableMap-0.1 ... linking ... done. 

*Conductive> [] 

[2] jj:bash* "snare" 16:50 29-Dec-IB 


Figure 1: A screenshot of Conductive in use 


expressivity, speed, static type system, large 
number of libraries, and ability to be either in¬ 
terpreted or compiled. It lacked a library suit¬ 
able for this author for realtime manipulation of 
musical processes. McClean is also developing 
Tidal [McLean and Wiggins, 2010], a Haskell 
library with a similar aim. 

3 The Solution 

A Haskell library called Conductive was cre¬ 
ated. It contains abstractions for musical time, 
musical events, and event loops. This gives the 
Haskell interpreter the ability to function as a 
code-based realtime sequencer for any output 
targets which can be connected to the system. 
Conductive does not aim to be an audio lan¬ 
guage, but a controller for audio output targets. 
The user is free to choose any OSC-aware out¬ 
put target, and this library is proposed as a way 
to coordinate those outputs. Another way to 
think of it is as a shell or scripting environment 
for realtime music. 

A library for getting, composing, and send¬ 
ing messages to JackMiniMix, an OSC-based 
mixer for JACK developed by Nicholas Hum- 
frey [Humfrey, 2005], was created 2 . 

A simple terminal-based clock visualization 
was also created. 


2 http://www.renickbell.net/doku.php?id=jackminimix 


beats per elapsed elapsed 

clock name tempo measure musical time clock time 


default 120.0bpm 4 49.3.001 1:39.50 


measure beat minutes seconds 

Figure 2: The terminal clock display 

4 Utilized Tools from Other 
Developers 

Before explaining the details of Conductive, it 
is necessary to list the components it was inte¬ 
grated with. The Glasgow Haskell Compiler In¬ 
terpreter (ghci) [Peyton Jones et al., 1992] was 
the core component used for executing Haskell 
code. Code was composed in vim [Moolenaar, 
2011], and sent to ghc via the tslirne plugin 
[Coutinho, 2010]. For OSC communication, 
Rohan Drape’s hose library was used [Drape, 
2010]. Output targets used were sesynth, the 
synthesizer component of SuperCollider [Mc¬ 
Cartney, 2010], and JackMiniMix. Drape pro¬ 
vides a library for communicating with sesynth 
via OSC called hsc3 [Drape, 2009]. 

5 Conductive in Detail 
5.1 Overview 

This library exists to wrap concurrent process 
manipulation in a way that makes controlling 
their timing more intuitive for musicians. At the 
same time, the library aims at being as concise 
as possible to lessen the burden on the user. 













The core components of the library are the 
data structures Player and MusicalEnvironment 
and a set of functions using these data struc¬ 
tures. A user designs a set of functions carrying 
out musical actions, such as playing a note on a 
synth or adjusting the parameter of synth. The 
user defines TempoClocks which have a tempo 
and time signature. The user also defines func¬ 
tions, called IOI (interonset interval) functions, 
describing how long to wait between executions 
of actions. These functions are stored in a Mu¬ 
sicalEnvironment. A Player is a collection of 
one action function, one IOI function, and one 
TempoClock and other related data. A Player 
is put into an event loop in which actions are 
executed after every defined IOI by using the 
play function. 

Conceptually, it has similarities with the con¬ 
cepts in SuperCollider of Routines, Tasks, and 
Patterns. Some key similarities and differences 
are noted below, along with details on each of 
these components. 

5.2 TempoClock 

The tempo is part of a TempoClock, a concept 
from SuperCollider which is reimplemented here 
in Haskell. A TempoClock is like a metronome 
keeping the current tempo but also contain¬ 
ing information about time signature and when 
tempo or time signature has been changed. 

A TempoClock is a record of the time the 
clock was started, a list of TempoChanges, and 
a list of TimeSignature changes. This allows a 
user to independently manipulate both tempo 
and time signature and to use these for com¬ 
posing and performance in addition to regular 
POSIX time. 

TempoClocks are stored in the MusicalEnvi¬ 
ronment. 

5.3 Players 

A data structure called a Player was designed as 
a way to sequence 10 actions. Players contain 
references to particular data which is stored in 
the MusicalEnvironment. The collection of data 
referenced by the Player results in a series of ac¬ 
tions being produced once the Player is played. 
This data consists of: 

• the name of the Player 

• its status (stopped, playing, pausing, 
paused, stopping, resetting) 

• a counter of how many times it has run an 
action 


player 


references 

- tempoClock 

- action function 

- IOI function 

- currentBeat 

- etc. 


Figure 3: Player: a data structure filled with 
references 

• which clock it is following 

• which IOI function it is using 

• which action function it is using 

• which interrupt function it is using 

• which beat its next action occurs on 

• which beat it started on 

• the POSIX time at which it was last paused 

An action function is a function that describes 
an event. An action function outputs a value 
of the 10 unit type. This basically means some 
kind of side effect is produced without return¬ 
ing a value like a double or a string. In practical 
terms, this could be a synthesis event, a param¬ 
eter change, or the action of playing, pausing, 
or stopping other Players or itself. It is thought 
that the user would use functions which send 
OSC messages to connected OSC-aware appli¬ 
cations. The action named in the Player can 
only take two parameters: the Player trigger¬ 
ing the action and the MusicalEnvironment it 
should read from. Beyond that, the design of 
the action is left to the user. A user might pre¬ 
fer to have many Players with simple actions, a 
few Players with complex actions, or some other 
combination. 

A fundamental concept is that of the time in¬ 
terval between the start times of two events, or 
interonset interval (IOI). [Parncutt, 1994] Su- 
perCollider refers to this as “delta” with regard 
to Patterns or “wait” for Routines. The IOI is 
defined in beats, and the actual time between 
events is calculated using the IOI value and the 



TempoClock referenced by the Player it is as¬ 
sociated with. IOI functions should also be de¬ 
signed to read the data from a Player and a Mu- 
sicalEnvironment. They can be designed in any 
way the user desires, including always return¬ 
ing a particular value, stepping through a list 
of values stored in a list somewhere, randomly 
choosing a value, or anything else the composer 
can imagine. 

An interrupt function is a function which is 
run once every time the play loop runs. It is 
useful for debugging purposes, and may be used 
to trigger other actions, such as stopping the 
player on a condition. 

Players bear some resemblance to Tasks or 
Patterns in SuperCollider; they can be played, 
paused, and stopped to produce music. How¬ 
ever, while Patterns in sclang can produce 
streams of any kind of data, Players in Con¬ 
ductive are designed to produce streams of side 
effects. While the data in a Pbind in SuperCol¬ 
lider is generally private [Harkins, 2009], all the 
data contained by a Player is visible. 

Players are stored in the Player store, a mu¬ 
table key-value store where the keys are Player 
name strings and the values are the Players 
themselves. This in turn is part of the Mu- 
sicalEnvironment. How patterns are stored in 
SuperCollider is up to the individual user. This 
library provides a readymade structure for that 
purpose. 

5.4 MusicalEnvironment 


MusicalEnvironment 


IOI 

function store 


IOI 

(delta) 

functions 


action function 
store . 


action 

functions 


tempoClock 

store .tempoClocks 


Figure 4: MusicalEnvironment: a data struc¬ 
ture for storage 

The MusicalEnvironment is a place to store 


data which is used by any of the user-initiated 
event loops. This data consists of: 

• the name of the environment 

• a store of Players 

• a store of TempoClocks 

• a store of IOI functions 

• a store of action functions 

• a store of interrupt functions 

5.5 Play 


play function gets 
current action function 


play runs 
IOI function 
and waits 


arguments 

P'ay , . 

a function / \ 

musical player name 

environment 


> 


action is 
forked to 
new thread 


play function gets 
current IOI function 


Figure 5: The play event loop 


The play function starts a thread which forks 
other processes according to a schedule deter¬ 
mined by the IOI function referenced in the 
Player. It takes a MusicalEnvironment, a 
Player store, and a Player name as arguments. 
First, the play function checks which action is 
referenced in the Player. It retrieves that func¬ 
tion from the MusicalEnvironment and forks it 
to a thread. It then checks which IOI function 
is referenced in the Player. It runs that func¬ 
tion and receives a numeric value specifying how 
long to wait in terms of beats. It then corrects 
that amount for jitter and sleeps for the cor¬ 
rected length of time. When the thread wakes 
up, the loop — checking the action and so on 
— repeats. 

It produces roughly similar results to calling 
play on a Pattern in SuperCollider in that it 
begins a process; however it is structured differ¬ 
ently. 

The problem of dealing with the delays in 
scheduled events is significant. Because various 
processes, including garbage collection, can con¬ 
ceivably interfere with correct timing, correc¬ 
tion of jitter is included in the play event loop. 
This library does not introduce a novel method 
for managing such delay, but rather adopts a 
design from McLean [McLean, 2004], An event 
intended to occur at time x actually occurs at 
time x + y, where y is the amount of time by 











which the event is late. The next event is sched¬ 
uled to occur at time x + z, where z is the IOI, 
so to account for the jitter, the wait time is set 
for x + (z-y). In practice, this delay is generally 
permissible for control data, while it would not 
be appropriate for audio data. 

The number of simultaneous play event loops 
is limited only by the memory and CPU of the 
host machine. Since at every loop the data used 
is refreshed, they can be manipulated in real 
time by changing the data stored in the Player 
or MusicalEnvironment. Which action function 
or IOI function is referenced in a Player can be 
changed. The action functions or IOI functions 
themselves can be modified. Any of the other 
data in the Players or MusicalEnvironment can 
be changed. By changing this data, the result¬ 
ing musical output can be changed. It is in this 
manner that a livecoded musical performance is 
realized. 

Such manipulation results in many threads 
and the potential exists for one thread to be 
writing data which is accessed by another. One 
problem of real-time multi-threaded systems is 
guaranteeing the thread safety of data. Haskell 
provides safe concurrency in the standard li¬ 
braries of the Glasgow Haskell Compiler (GHC). 

5.6 An Example of How Players Work 

Here is an example of how Players work, shown 
in figure 6. 

Consider a timpani Player called “A” who has 
only two jobs. The first job is to hit the timpani. 
The second job is to wait for a given amount of 
time, like that written on a score. He hits the 
timpani, then he waits, then he hits the timpani 
again and waits, in a loop until he is asked to 
stop. Now imagine that this Player is joined 
by another: Player “B”. The second Player has 
only two jobs. The first is to adjust the tuning 
of the timpani; the second job is the same as 
that of the first Player. He tunes the timpani 
and waits, and then tunes it again and waits, 
repeating like the first Player. 

The first timpani Player is a Player stored un¬ 
der the key “A” in the Player store. Its action 
function is “hit the timpani”, which may corre¬ 
spond to triggering a synthdef on scserver called 
“timpani”, which results in a timpani sound be¬ 
ing played. The second Player is called “B”, and 
its action function, “tune timpani”, is to change 
the frequency parameter used by the “hit the 
timpani” function. Each of them has its own 
IOI function. 


Let’s expand the situation to include two 
more Players, Players “C” and “D”, who corre¬ 
spond to Players “A” and “B” but are involved 
with another timpani. The resulting sound is 
two timpanis being played at the same time. In 
this case, the “hit the timpani” action is de¬ 
signed to use the name of the Player to deter¬ 
mine which frequency should be used. In the 
same way, the “tune timpani” function uses the 
Player name to determine which frequency it is 
tuning and which frequency to tune to. 

Now, interestingly, we’ll add a fifth Player, 
who is starting and stopping the Players above. 
Its action function cycles through a list of ac¬ 
tions. Its first action is to start Player “A”. Its 
second action is to start Player “B”. Its third 
action could be to start Players “C” and “D” 
simultaneuously. Its fourth action could be to 
pause Players “A”, “B”, and “D”. The design 
of any action is up to the intentions of the mu¬ 
sician. 



5.7 Code Examples of Conductive 
Usage 

A rudimentary sample of usage and correspond¬ 
ing code is given below. 

First, the relevant Haskell modules must be 



imported, which is accomplished by loading a 
Haskell document containing the necessary im¬ 
port statements. 

:load Conductive.hs 

This example uses SuperCollider, so a conve¬ 
nience command which sets up a group on sc- 
server is called. 

defaultSCGroup 

A default MusicalEnvironment is instantiated. 
It is assigned to the variable “e”. 

e <- defaultMusicalEnvironment 

An scserver-based sampler is instantiated using 
this command, which also creates the necessary 
Players and action functions in the MusicalEn¬ 
vironment. The function takes a path and the 
MusicalEnvironment as arguments. 

s <- initializeSampler "../sounds/*" e 

All of the Players in a specified MusicalEnviron¬ 
ment can be started with the play All function. 
The argument, like above, is the MusicalEnvi¬ 
ronment. 

playAll e 

The status of all the players in a specified Mu¬ 
sicalEnvironment can be viewed with the dis¬ 
play Players command. 

displayPlayers e 

A list of players can be paused using the pauseN 
function. The specified players will be looked up 
in the MusicalEnvironment. 

pauseN e ["samplerl","sampler2"] 

Those players could be restarted at a specified 
time, in this case the first beat of the 16th mea¬ 
sure, using the playNAt function. The string 
after “e” is the specified time, given in terms of 
measure and beat. 

playNAt e "15.0" ["samplerl","sampler2"] 

The tempo of a particular TempoClock can be 
changed with the changeTempo function. The 
string “default” is the name of the TempoClock 
that is to be manipulated. 


changeTempo e "default" 130 

A new IOI function can be created. This func¬ 
tion call gives the name “newIOI” to an IOI 
function which will be stored in the MusicalEn¬ 
vironment. That string is followed by the offset, 
the number of beats before the first event takes 
place. The list contains IOI values; in this case, 
an interval of three beats passes between the 
first two events. 

newIOIFunctionAndlOIList e "newIOI" 

0 [3,0.25,1,0.5,2,0.25,3] 

A player can be told to use this new IOI function 
by calling the swapIOI function. After specify¬ 
ing the MusicalEnvironment, the name of the 
player and the name of the IOI function are 
given. 

swapIOI e "sampler2" "newIOIPattern" 

All of the players can be stopped with the 
stopAll function. 

stopAll e 

6 Conclusion and Future Directions 

Rudimentary livecoding performances were 
made possible. The timing was found to be 
adequate for musical performances, though mil¬ 
lisecond timing errors remain. While the library 
was sufficient for very basic performances, it was 
necessary to create additional libraries for con¬ 
trol and algorithmic composition to achieve a 
usable interface and more sophisticated perfor¬ 
mances. 

This library alone was far from sufficient to 
replace current GUI sequencers for most users, 
though it is hoped this is a good foundation for 
further research in this direction. 

An evaluation method to quantify the usabil¬ 
ity of this approach should be considered. Ad¬ 
ditionally, determining the performance of this 
system versus sclang, Impromptu and others 
may be valuable. 

The library will be tested in performance sit¬ 
uations and expanded to be a more complete 
integrated development environment and per¬ 
formance tool for livecoding performers. Its use 
in other real-time music applications will also 
be tested. 

The jitter described above is believed to be 
at least in part due to the garbage collection 
routines of GHC. Improvements to the GHC 



garbage collector are currently being made by 
its developers. [Marlow, 2010] It is hoped that 
the gains they make will carry over positively to 
the performance of this system in terms of re¬ 
duced delays. There could be other contributing 
factors, but they have not yet been identified. A 
deeper investigation into potential causes of jit¬ 
ter and their solutions needs to be undertaken. 

Another serious problem involves tempo 
changes. If the tempo is changed while a play 
process is sleeping, the next event in that pro¬ 
cess will be out of sync: early if the tempo is 
reduced, or late if the tempo is increased. Fol¬ 
lowing events, however, will occur at the correct 
times. This is because the function for awak¬ 
ening the sleeping Player is unaware of tempo 
changes and thus cannot adjust the time accord¬ 
ingly. A revised method for sleeping threads 
which is tempo-aware should be developed. 

An important next step is developing a li¬ 
brary to make it easy to use AUDI devices with 
Conductive. 

Use of this library by visually-impaired users 
should be examined, as this text interface may 
offer such users increased usability. It will be 
necessary to find users with a braille display and 
familiarity with vim or ernacs for usability test¬ 
ing. 
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